Digital Media Methods

ABSTRACT

A consumer who encounters music of interest can operate a portable device to obtain a display of associated meta data (e.g., the title and artist, the artist&#39;s concert schedule, etc.) obtained from a remote computer system. The consumer can then initiate an electronic commerce transaction from the device, e.g., ordering a copy of a song. The song, or a link thereto, can be sent to the consumer (either at the originating device, or a separate device), or can be sent to another person designated by the consumer. A variety of other technologies and arrangements are also detailed.

RELATED APPLICATION DATA

This application is a division of co-pending application Ser. No.09/574,726, filed May 18, 2000, which claims priority to application60/134,782, filed May 19, 1999.

The subject matter of the present application is related to thatdisclosed in application Ser. No. 09/234,780, filed Jan. 20, 1999 (acontinuation of which is now U.S. Pat. No. 6,332,031; Ser. No.09/314,648, filed May 19, 1999 (now U.S. Pat. No. 6,681,028); Ser No.09/337,590, filed Jun. 21, 1999; Ser. No. 09/433,104, filed Nov. 3, 1999(now U.S. Pat. No. 6,636,615); Ser. No. 09/441,819, filed Nov. 17, 1999(now U.S. Pat. No. 6,539,095); Ser. No. 09/441,821, filed Nov. 17, 1999(now U.S. Pat. No. 6,363,159); Ser. No. 09/442,441, filed Nov. 17, 1999(now U.S. Pat. No. 6,587,821); 09/464,307, filed Dec. 15, 1999 (now U.S.Pat. No. 6,286,036); Ser. No. 09/473,075, filed Dec. 28, 1999 (now U.S.Pat. No. 6,560,349); Ser. No. 09/476,686, filed Dec. 30, 1999; Ser. No.09/482,752, filed Jan. 13, 2000 (now U.S. Pat. No. 6,330,335); Ser. No.09/484,742, filed Jan. 18, 2000 (now U.S. Pat. No. 6,983,051); Ser. No.09/498,223, filed Feb. 3, 2000 (now U.S. Pat. No. 6,574,350);60/180,364, filed Feb. 4, 2000; Ser. No. 09/520,406, filed Mar. 8, 2000(now U.S. Pat. No. 6,266,430); Ser. No. 09/563,664, filed May 2, 2000(now U.S. Pat. No. 6,505,160); and Ser. No. 09/562,517, filed May 1,2000.

FIELD OF THE INVENTION

The present invention relates to use of, and/or commerce in, electronicmedia content, such as digital music and video.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing the participants, and channels, involved inthe distribution of music.

FIG. 2 shows a conceptual model of how music artists, record labels, andE-Music distributors can all interact with a Media Asset ManagementSystem, of which several are detailed in the following specification.

DETAILED DESCRIPTION

For expository convenience, much of the following discussion focuses onmusic, but the same principles and techniques are largely or whollyapplicable to other source data, whether non-music audio, video, stillimagery, printed materials, etc.

Music Asset Management

Referring to the figures, the music distribution process begins with acreative artist 10. The artist's music has traditionally beendistributed by a record label 12. (While the following discussion refersto distribution through such a label, it should be understood that suchdistribution can just as well be effected directed under the artist'scontrol, without a record label intermediary.)

In traditional distribution 14, the record label produces tangiblemedia, such as records, tapes, videos (e.g. music videos), and CDs 16.These media are physically distributed to end-consumers 18.Additionally, the label 12 distributes the music media to outlets 20,such as radio and TV stations, cable and satellite systems, etc., whichbroadcast (or narrowcast) the artist's work to an audience. Distributionthrough such media outlets may be monitored by playout trackingservices. Playout tracking data, collected by firms including Arbitron,Nielsen, ASCAP, BMI, etc., can be used to compute royalty payments, toverify broadcast (e.g. for advertising), etc.

Increasingly, the distribution of the music to the media outlets isperformed electronically. Such distribution first took the form ofanalog audio over high quality landlines or satellite channels. Digitalaudio quickly supplanted analog audio in such distribution channels dueto higher fidelity.

More recently, distribution of the music from the record labels to themedia outlets has occurred over secure links, now including theinternet. Such security was first provided simply by scrambling theaudio signal or data. More sophisticated “container”-based systems arenow coming into vogue, in which the audio is “packaged” (often inencrypted form) with ancillary data.

Electronic distribution of music to the consumer is also gainingpopularity, presently in the MP3 format primarily. The music providersmay deal directly with the public, but more commonly effect suchconsumer distribution through a newly emerging tier of digital mediaoutlets, such as internet sites that specialize in music. From suchsites, consumers can download digital audio files into personal digitalaudio players. (The Diamond Rio, and the Audible MobilePlayer devicesare some of the first of what will doubtless be a large number ofentrants into this personal internet audio appliance market.) Or thedownloaded data can be stored by the consumer-recipient onto any otherwriteable media (e.g. hard disk, CD, DVD, tape, videotape, etc.).Typically a personal computer is used for such downloading, but thisintermediary may be dispensed with by coupling next generation ofpersonal audio appliances to an internet-like link.

The data downloaded by the consumer can be stored either in the nativedigital format, translated into another digital format (whichtranslation may include decryption), converted into analog and recordedin analog form, etc.

Unauthorized copying or use of the music can occur anywhere in theforegoing channels. However, one of the greatest risks occurs once themusic has been delivered to the consumer (whether by tangible media, bytraditional broadcast media outlets, by emerging digital distribution,or otherwise).

The general idea of embedding auxiliary data into music (i.e.watermarking) has been widely proposed, but so far has been of limitedapplicability.

For example, GoodNoise is planning to embed a digital signature—termed amultimedia identifier, or MMI—in its MP3 music. MMI will register thesong and its author with a licensing number. In addition to providinginformation about the songwriter and distributor, this digital encodingmay also include lyrics, liner notes, and other information. But all ofthe proposed uses serve only to convey information from the distributorto the consumer; use for “tracking” is actively disclaimed. (Wired News,“GoodNoise Tags MP3 Files,” Feb. 3, 1999.)

The Genuine Music Coalition—a partnership of various companies in themusic distribution business—likewise has announced plans to employwatermarking of MP3 music. The watermarking technology, to be providedby Liquid Audio, will convey data specifying the artist or producercontact, copyright data, and a number to track ownership. The Coalitionhopes that the provision of this embedded information will help thwartpiracy. Industry observers believe Liquid Audio will next introduceplayback technology only plays audio in which its watermark is detected.(Wired News, “Liquefying MP3,” Jan. 23, 1999.)

A similar initiative has been announced by the Recording IndustryAssociation of America (RIAA). Termed the Secure Digital MusicInitiative (SDMI), the program seeks to define a voluntary specificationthat will assure proper compensation to those who produce and distributemusic. One element of the system will likely be a watermarkingcomponent. (Dow Jones Newswire, “Spurred By Maverick Technology, MusicIndustry Eyes Web,” Dec. 31, 1998.)

Yet another initiative has been announced by Solana and ASCAP. Othercompanies promoting watermarking for music include Aris Technology,MCY.com, and AudioSoft.

The watermark payload can represent various types of data. An exemplarypayload includes data relating to the artist, distribution entity,title, and copyright date/proprietor. Additionally, the payload caninclude a digital object identifier—an ISBN-like number issued by acentral organization (e.g. a rights management organization) to uniquelyidentify the work.

Such payload data can be encoded literally (e.g. the title by a seriesof ASCII characters, etc.). In other embodiments, codes or abbreviationscan be employed—with each code having a known meaning. In still otherembodiments, the data can be meaningless by itself, but may serve as akey (e.g., a Unique Identifier, or UID) into a remote data database orrepository. An example of such a remote data repository is a web site ata Master Global Address (MGA) associated with content, as detailedbelow.

An exemplary data payload may, for example, have the following format: AB C D E F G H IWhere A is a six-byte (8-bits to a byte) ASCII string serving as adigital object identifier (which may serve as a link to a Master GlobalAddress through a default name server, as discussed below), B is atwo-byte ASCII field serving as a key into an “artist” field of theremote database, C is a three-byte ASCII field serving as a key into a“title” field of the remote database; D is a 14-bit field serving as akey into a “label” field of the remote database, E is an 8-bit integerrepresenting the work's year of first publication (with 0 representingthe year 2000); F is a 10-bit field serving as a key into a “price”field of the remote database, G is a two-byte usage control string(detailed below), H is a streaming data channel, and I is a string ofbits serving as a cyclic redundancy checksum for the foregoing. (Moresophisticated error correcting checksums can, of course, be employed.)This payload format totals 136 bits, exclusive of the CRC coding and thestreaming data channel.

This payload is encoded repeatedly, or redundantly through the music, sothat the full payload can be decoded from partial excerpts of the music.

The encoding is also desirably perceptually adaptive, so that higherenergy encoding is employed where the listener is less likely toperceive the additional “noise” introduced by the encoding, and viceversa. Various techniques for perceptually adaptive encoding are known.For example, some tie the amplitude of the encoded signal to theinstantaneous amplitude of the music. Others exploit psychoacoustic“masking” of one signal by a spectrally- or temporally-adjoining signalof higher energy. Still other approaches fill gaps in the music'sspectrum with watermark energy.

In other embodiments, perceptually adaptive encoding is not used. Insome such embodiments, no tailoring of the temporal or spectralcharacteristics of the watermark signal is employed. In others, thewatermark signal is spectrally filtered to emphasize low frequency audiocomponents (e.g. less than 500 Hz), high frequency audio components(e.g. higher than 2500 Hz), or mid-frequency audio components (500-2500Hz).

The streaming data field channel (H) is a medium by which data can beconveyed from a distribution site (or other site) to the end user. Suchdata may be entirely unrelated to the underlying work. For example, itmay serve a utilitarian purpose, such as conveying data to a memory inthe consumer device to replace previously-stored data that isout-of-date. It may be a commercial channel on which bandwidth is soldfor access to the consumer or the consumer's device. Essentially anypurpose can be served by this streaming data field. Unlike most of theother fields, the streaming data field may not endlessly repeat the samedata, but can convey data that changes with time.

Desirably, the encoding is performed in a manner permitting recovery ofthe watermark data even if the audio is corrupted, e.g. by formatconversion, re-sampling, tape wow and flutter, compression, coding, orvarious forms of audio processing (e.g. filtering, pre-emphasis,re-scaling, etc.). One way to provide for such robustness is to encode asignal of known character that can be recognized through all suchcorruption. By identifying such known signal, the watermark signal canthen be decoded. (The known signal can take various forms, e.g. asynchronization signal, a marker signal, calibration signal, a universalcode signal as described in applicant's patents, etc.)

In some embodiments, a watermark “dial-tone” signal is provided. Thisdial-tone signal is a low amplitude, relatively wideband, repetitivesignal that commonly conveys only limited information (e.g. a single bitof information). Its presence in an audio signal can serve as a “do notrecord,” or similar instruction signal. Alternatively, or in addition,the dial-tone signal can serve as an aid in “locking” to a plural-bitdigital watermark signal that is also encoded in the audio. For example,the cyclical repetition of the signal can serve to identify the start ofthe plural-bit digital watermark signal. Or the spectrum or repetitionrate of the signal can identify any temporal corruption of the audio. Anexemplary such signal is detailed as a “simple universal code” in U.S.Pat. No. 5,636,292.

A track of music can be pre-authorized for specified types of use. Forexample, the usage control string of the watermark payload may include asix-bit field detailing the classes of devices for which the audio isauthorized. Each bit would correspond to a different class of device.Class 1 devices may be personal playback devices with only analog-audiooutput. Class 2 devices may be personal entertainment devices capable ofoutputting music in digital (e.g. MP3, redbook, *.WAV) format, as wellas analog audio. Class 3 devices may be personal computer systems (i.e.with essentially unlimited ability for processing and outputting digitalaudio). Etc., etc. A device to which such MP3 audio is provided wouldcheck the usage control string data to determine whether it isauthorized to utilize the audio. A personal playback device withanalog-only output, for example, would examine the first bit of theusage control string. If it was “1,” the device would be authorized touse (i.e. playback) the MP3 data; if it was a “0,” the device wouldrefuse to play the music.

In addition to pre-authorization for certain classes of devices, theusage control string can also include bits indicating the number ofpermitted playbacks. This data can be encoded in bits seven throughnine, representing eight possibilities:

0—no playback permitted

1—single playback permitted

2—two playbacks permitted

3—three playbacks permitted

4—four playbacks permitted

5—five playbacks permitted

6—10 playbacks permitted

7—unlimited playbacks permitted

8—refer to associated data (within the watermark, or stored at a remotesite) which specifies number of permitted playbacks.

The playback device may include a non-volatile store in which the numberof permitted playbacks is stored for each track of music. The devicewould decrement this number at the beginning of each playback.

The usage control string can also include a two-bit field (bits ten andeleven) indicating recording permissions. A value of 0 means that datacorresponding to the MP3 audio (regardless of digital format) shouldnever be made available to another digital device. A value of 1 meansthat the data corresponding to the MP3 data may be made available onceto another digital device. A value of 2 means that the data may be madeavailable an unlimited number of times to other digital devices. (Value3 is reserved.)

Another data field that can be included in an audio watermark is arating that indicates age-appropriateness. Music with violence or sexualthemes might be given a rating akin to the MPAA “PG-13” or “R” rating.Audio appliances may be programmed to recognize the rating of incomingmusic, and to interrupt playback if the rating exceeds a certainthreshold setting. Various known techniques can be employed to assurethat such settings cannot readily be changed, e.g., by juvenilelisteners.

Another data field that can be included in an audio watermark is a datefield. This field can indicate either the date the music waswatermarked, or a date in the future on which certain rights associatedwith the music should change. Some consumers, for example may not wishto purchase perpetual playback rights to certain musical selections. Theright to play a selection for 6 months may suffice for many consumers,especially if the price is discounted in view of the limited term. Suchan arrangement would not be wholly disadvantageous to musicdistributors, since some consumers may end up purchasing music twice iftheir initial assessment of a musical selection's appeal was tooshort-sighted. (Naturally, the playback equipment would require a sourceof real-time clock data against which the date field in the watermarkcan be checked to ensure that the playback rights have not yet expired.)

Another of the data fields that can be included in an audio watermarkspecifies technical playback parameters. For example, the parameter cancause the playback appliance to apply a spectral equalization thatfavors bass frequencies, or treble frequencies, or mid-rangefrequencies, etc. Other pre-configured equalization arrangements cansimilarly be invoked responsive to watermark data. Likewise, theparameter can invoke special-effects provided by the playback appliance,e.g., echo effects, reverb, etc. (Again, such parameters are usuallyrepresented in an abbreviated, coded form, and are interpreted inaccordance with instructions stored in a memory (either in the playbackappliance, or linked thereto).

The same data fields and principles can be applied to non-audio content.In video, for example, watermarked data can adaptively control thedisplay monitor or playback parameters (e.g., color space) to enhancethe viewing experience.

Music Asset Management/Commerce

The majority of domestic music piracy is not organized. Rather, it is acrime of opportunity and convenience. If the crime were made moredifficult, the alternative of obtaining a copy through legitimatechannels would be less onerous. Similarly, if the procedure forobtaining a copy through legitimate channels were simplified, theincentive for piracy would be reduced. Watermarking facilitatesboth—making the crime more difficult, and making legitimate musicacquisition easier.

Consider, for example, the pricing of music in conventional recordstores. A CD (compact disk) may cost $15, but its sale may be driven byjust one or two popular songs on the disk. To obtain these songs, theconsumers must purchase the entire disk, with perhaps a dozen songs ofno particular interest. This, in essence, is a tying arrangement thatbenefits the record labels while prejudicing the consumers. Given thesecircumstances, and a ready opportunity to make copies, it is notsurprising that customers sometimes make illicit copies.

One classic technique of avoiding purchase of a complete collection ofmusic, when only one or two songs is desired, is to record the music offthe radio. While of dubious legality, this technique was popular in theera of combined cassette/radio players. However, the desired music wassometimes difficult to encounter in a radio broadcast, and the qualitywas less than superb.

The combined cassette/radio player has now evolved into a generalpurpose computer with wide-ranging functionality, and othersophisticated devices. Music can be acquired off the web, and can berecorded in various forms (e.g. in a personal MP3 player, stored on ahard disk, stored on a writeable CD-ROM, played back and recorded onanalog cassette, etc., etc.). The quality can be quite high, and theerratic broadcast time problems of radio broadcasts have been overcomeby the web's on-demand delivery mechanisms. (Moreover, the music can bedownloaded in faster-than-realtime, a further benefit overrecording-off-the-air techniques.) One hybrid between the new and old isa novel radio (e.g., for use in a car) that has a “capture” button onthe front panel (or other form of user interface, e.g., a Capture iconon a GUI). If a user hears a song they want to record and keep, theypress the Capture button while the song is playing. In response, theradio device decodes a watermark embedded in the music, and therebyknows the identity of the music. The radio then makes a wirelesstransmission identifying the user and the desired song. A local repeaternetwork picks up the wireless signal and relays it (e.g. by wirelessrebroadcast, by modem, or other communication medium) to a musicclearinghouse. The clearinghouse charges the user a nominal fee (e.g.via a pre-arranged credit card), and queues the music for download to apredetermined location associated with the user.

In one embodiment, the predetermined location is the user's owncomputer. If a “live” IP address is known for the user's computer, themusic can be transferred immediately. If the user's computer is onlyoccasionally connected to the internet, the music can be stored at a website (e.g. protected with a user-set password), and can be downloaded tothe user's computer whenever it is convenient.

In other embodiments, the predetermined location is a personal musiclibrary maintained by the user. The library can take the form, e.g., ofa hard-disk or semiconductor memory array in which the user customarilystores music. This storage device is adapted to provide music data toone or more playback units employed by the user (e.g. a personal MP3player, a home stereo system, a car stereo system, etc.). In mostinstallations, the library is physically located at the user'sresidence, but could be remotely sited, e.g. consolidated with the musiclibraries of many other users at a central location.

The personal music library can have its own internet connection. Or itcan be equipped with wireless capabilities, permitting it to receivedigital music from wireless broadcasts (e.g. from the clearinghouse). Ineither case, the library can provide music to the user's playbackdevices by short-range wireless broadcast.

By such arrangement, a user can conveniently compile an archive offavorite music—even while away from home.

Many variants of the foregoing are of course possible. The radio can bea portable unit (e.g. a boombox, a Walkman radio, etc.), rather than anautomotive unit. The UI feature employed by the user to initiate capturea musical selection need not be a button (physical or on-screen). Forexample, in some embodiments it can be a voice-recognition system thatresponds to spoken commands, such as “capture” or “record.” Or it can bea form of gesture interface.

Instead of decoding the watermark only in response to the user's“capture” command, the radio can decode watermarks from all receivedprograms, and keep the most recent in a small FIFO memory. By sucharrangement, the user need not issue the capture instruction while thesong is playing, but can do so even after the song is finished.

In some embodiments, data corresponding to the watermark can be madeavailable to the user in various forms. For example, it can be presentedto the user on an LCD screen, identifying the artist and song currentlyplaying. If a corresponding UI button is activated, the device canso-identify the last several selections. Moreover, the data need not bepresented to the user in displayed form; it can be annunciated by knowncomputer-speech technologies instead.

In embodiments in which the watermark does not convey ASCII text data,but instead conveys UIDs, or coded abbreviations, the device mustgenerally interpret this data before presenting it to the user. In anillustrative embodiment, the device is a pocket-sized FM radio and isequipped with a 1 megabyte semiconductor non-volatile RAM memory. Thememory includes a data structure that serves as a look-up table,matching code numbers to artist names and song titles. When the userqueries the device to learn the identify of a song, the memory isindexed in accordance with one or more fields from the decodedwatermark, and the resulting textual data from the memory (e.g. songtitle and artist) is annunciated or displayed to the user.

In most applications, such memory will require frequent updating. The RFreceiver provides a ready mechanism for providing such updated data. Inone embodiment, the radio “awakens” briefly at otherwise idle momentsand tunes to a predetermined frequency at which updated data for thememory is broadcast, either in a baseband broadcast channel, or in anancillary (e.g. SCA) channel.

In variants of the foregoing, internet delivery of updated memory datacan be substituted for wireless delivery. For example, the artist/songtitle memory in the personal player can be updated by placing the playerin a “nest” every evening. The nest (which may be integrated with abattery charger for the appliance) can have an internet connection, andcan exchange data with the personal device by infrared, inductive, orother proximity-coupling technologies, or through metal contacts. Eachevening, the nest can receive an updated collection of artists/songtitles, and can re-write the memory in the personal device accordingly.By such arrangement, the watermark data can always be properlyinterpreted for presentation to the user.

The “Capture” concepts noted above can be extended to other functions aswell. One is akin to forwarding of email. If a consumer hears a songthat another friend would enjoy, the listener can send a copy of thesong to the friend. This instruction can be issued by pressing a “Send”button, or by invoking a similar function on a graphical (or voice- orgesture-responsive) user interface. In response, the applianceso-instructed can query the person as to the recipient. The person candesignate the desired recipient(s) by typing in a name, or a portionthereof sufficient to uniquely identify the recipient. Or moretypically, the person can speak the recipient's name. As is conventionalwith hands-free vehicle cell phones, a voice recognition unit can listento the spoken instructions and identify the desired recipient. An“address book”—like feature has the requisite information for therecipient (e.g., the web site, IP address, or other data identifying thelocation to which music for that recipient should stored or queued, theformat in which the music should be delivered, etc.) stored therein. Inresponse to such command, the appliance dispatches instructions to theclearinghouse, including an authorization to debit the sender's creditcard for the music charge. Again, the clearinghouse attends to deliveryof the music in a desired manner to the specified recipient.

Still further, a listener may query the appliance (by voice, GUI orphysical button, textual, gesture, or other input) to identify CDs onwhich the then-playing selection is recorded. Or the listener may querythe appliance for the then-playing artist's concert schedule. Again, theappliance can contact a remote database, relay the query, and forwarddata from the watermark payload identifying the artist and/or song titleto which the query relates. The database locates the requested data, andrelays same back to the appliance for presentation (via a display, bymachine speech, or other output) to the user. If desired, the user cancontinue the dialog with a further instruction, e.g., to buy one of theCDs on which the then-playing song is included. Again, this instructionmay be entered by voice, GUI, etc., and dispatched from the appliance tothe clearinghouse, which can then complete the transaction in accordancewith pre-stored information (e.g. credit card account number, mailingaddress, etc.). A confirming message is relayed to the appliance forpresentation to the user.

While the foregoing transactions require a link to a remote site ordatabase, other watermark-based consumer services can be providedwithout such a link. For example, a user can query the appliance as tothe artist or song-title of the selection currently playing. Theappliance can consult the embedded watermark data (and optionallyconsult a memory to determine the textual names associated with codedwatermark data), and provide the requested information to the user(e.g., by a display, annunciation, or other output).

The foregoing concepts (e.g. Capture, Send, etc.) can also be employedin connection with internet- rather than radio-delivery of music. (Thefollowing discussion is illustrated with reference to the “Capture”function, but it will be recognized that the other earlier-discussedfeatures can be similarly implemented.)

There are many commercial web sites that sell audio (in CD form orotherwise), and offer limited free music downloads, (or music clips) asan enticement to lure consumers. But there are also a great number ofmusic web sites that have no commercial pretense. They are hosted bymusic lovers strictly for the enjoyment of other music lovers. Whenmusic is downloaded from such a web site, the end-user's computer cananalyze the digital data to decode watermark data therefrom. Again, theuser can be presented with a “Capture” button that initiates acommercial transaction, by which a complete copy of the then-downloadedaudio is sent to a prearranged storage location, and the user's creditcard is debited accordingly. This transaction can occur independently ofthe site from which the music is downloaded (e.g. through theclearinghouse referenced above).

While the “Capture” button can be presented on the web-site, this wouldgenerally not be in keeping with the non-commercial nature of such websites. Instead, in an exemplary embodiment, the Capture feature is asoftware program resident at the user's computer. When this softwareprogram is invoked by the user, a socket channel is instantiated betweenthe user's computer and the clearinghouse over the then-existinginternet connection. The decoded watermark data and user ID istransmitted to the clearinghouse over this channel, without interruptingthe user's other activity (e.g. downloading music from thenon-commercial web site). In response, the clearinghouse transmits themusic to the prearranged location and attends to billing.

In some embodiments, a watermark detector is included as part of theoperating system, and constantly monitors all TCP/IP, or other internet,data received by the user's computer, for the presence of watermarks. Insuch case, when the Capture feature is invoked, the program examines amemory location in which the operating system stores the most-recentlyreceived watermark data. In another embodiment, the computer does notmonitor all internet traffic for embedded watermark data, but includesan API that can be called by the Capture program to decode a watermarkfrom the data then being received. The API returns the decoded watermarkdata to the Capture program, which relays same to the clearinghouse, asabove. In still another embodiment, the watermark decoder forms part ofthe Capture program, which both decodes the watermark and relays it tothe clearinghouse when the Capture program is invoked by the user.

There are various techniques by which the Capture program can beselectively invoked. One is by a keyboard macro (e.g. by a combinationof keyboard keys). Another is by a program icon that is always presentedon the screen, and can be double-clicked to activate. (Again,confirmation processes may be called for, depending on the likelihood ofinadvertent invocation.) Many other techniques are likewise possible.

In the just-contemplated scenario, the Capture operation is invokedwhile the user is downloading music from a non-commercial web site. Thisseems somewhat redundant, since the downloading—itself—is transferringmusic to the user's computer. However, the Capture operation providesadded value.

In the case of streaming audio, the audio is not typically stored in alocation in which it can be re-used by the consumer. It can belistened-to as delivered, but is then gone. Capturing the audio providesthe user a copy that can be played repeatedly.

In the case of downloaded music files, the music may have been encodedto prevent its recordal on other devices. Thus, while the user maydownload the music onto a desktop computer, copy-prevention mechanismsmay prevent use of that file anywhere else, e.g. on a portable musicappliance. Again, Capturing the audio provides the user a copy that canbe transferred to another device. (The music file provided by theclearinghouse can have copy-prevention limits of its own—e.g., the filecan be copied, but only once, or the file can be copied only ontodevices owned by the user.)

(Confirmation of device ownership can be implemented in various ways.One is to identify to the clearinghouse all music devices owned by auser at the time the user registers with the clearinghouse (supplementedas necessary by later equipment acquisitions). Device IDs associatedwith a user can be stored in a database at the clearinghouse, and thesecan be encoded into the downloaded music as permitted devices to whichthe file can be copied, or on which it can be played.)

The commerce opportunity presented by non-commercial music web-sites isbut one enabled by digital watermarks. There are many others.

To take one example, consider the media by which music and artists arepresently promoted. In addition to radio airtime, these include musicvideos (a la MTV), fan magazines, web advertisements, graphical icons(e.g. the Grateful Dead dancing bears), posters, live events, movies,etc. Watermarked data can be used in all such media as a link in acommercial transaction.

A poster, for example, typically includes a photo of the artist, and maycomprise cover-art from a CD. The photo/art can be digitally watermarkedwith various types of data, e.g., the artist's identify, the recordlabel that distributes the artist's work, the music project beingparticularly promoted by the poster (e.g. a CD, or a concert tour), afan web-site related to the artist, a web-site hosted by the recordlabel for selling audio in CD or electronic form, a web-site from whichfree music by the artist can be downloaded, data identifying the posteritself, etc.

A user, equipped with a portable appliance that merges the functions ofpalmtop computer and digital camera, can snap an image of the poster.The processor can decode the watermarked data, and initiate any ofvarious links based on the decoded data.

In an exemplary embodiment, after snapping the picture, the user invokesa software program on the device that exposes the various links gleanedfrom the snapped image data. Such a program can, for example, presentthe option of linking to the artist's fan web site, or downloading freestreaming audio or music clips, or ordering the promoted CD, orrequesting the above-noted clearinghouse to download a personal copy ofselected song(s) by the artist to the user's personal music library,etc. (The device is presumed to have a wireless internet link. Indevices not having this capability, the requested actions can be queuedand automatically executed when a link to the internet is available.)

Still more complex transactions can be realized with the use of a remotedatabase indexed by digital watermark fields decoded from the poster.For example, the poster may promote a concert tour. Fields of thedigital watermark may identify the artist (by a code or full text), anda web site or IP address. The user appliance establishes a link to thespecified site, and provides the artist identifier. In response, thesite downloads the tour schedule for that artist, for display on thedevice. Additionally, the downloaded/displayed information can include atelephone number that can be used to order tickets or, more directly,can indicate the class of seats still available at each (or a selected)venue, and solicit a ticket order from the user over the device. Theuser can supply requested information (e.g. mailing address and chargecard number) over the return channel link (wireless or wired, as thecase may be), and the ticket(s) will be dispatched to the user. In thecase of a wireless link, all of this can occur while the user isstanding in front of the movie poster.

Similar systems can be implemented based on watermark data encoded inany other promotional media. Consider music videos. Using knownTV/computer appliances, watermark data added to such videos can readilybe decoded, and used to establish links to audio download, CD-sales, fanclub, concert ticket outlet web sites, etc., as above.

Even live events offer such watermark-based opportunities. The analogaudio fed to public address or concert speakers can be watermarked(typically before amplification) to encode plural-bit digital datatherein. A next generation personal music appliance (e.g. one with awireless interface to the internet) can include analog record capability(e.g. a built-in microphone, analog-to-digital converter, MP3 encoder,coupled to the unit's semiconductor memory). A user who attends a liveevent may record an excerpt of the music. The watermark can then bedecoded, and the extracted data used to access the links and commerceopportunities reviewed above.

Cinema movies offer both audio and visual opportunities forwatermark-based commerce opportunities. Either medium can be encoded toconvey information of the types reviewed above. A personal appliancewith image- or audio-capture capabilities can capture an excerpt of theaudio or imagery, decode the watermark data therefrom, and perform anyof the linking, etc., functions reviewed above.

The consumer-interest watermarks reviewed above are only exemplary. Manyothers will be recognized as useful. For example, promotional clipspresented before a feature film presentation can include watermark datathat point (either by a literally encoded web address link, or by an IDcode that indexes a literal link in a remote link database) to reviewercritiques of the previewed movies. Watermark data in a featured filmpresentation can lead to web sites with information about the moviestars, the director, the producer, and can list other movies by each ofthese persons. Other watermark-conveyed web links can presentopportunities to buy the movie on videotape, to purchase the moviesoundtrack, to buy movie-related toys and games, etc.

More on Device Control

Much of the foregoing has focused on watermark encoding to provideenhanced customer experiences or opportunities. Naturally, watermarksdata can alternatively, or additionally, serve the interests of themedia owner.

To illustrate, consider watermarked music. The media owner would be bestserved if the watermark serves dual purposes: permissive andrestrictive. Permissively, music appliances can be designed to play (orrecord) only music that includes an embedded watermark signaling thatsuch activity is authorized. By this arrangement, if music is obtainedfrom an unauthorized source and does not include the necessarywatermark, the appliance will recognize that it does not have permissionto use the music, so will refuse requests to play (or record).

As noted, music appliances can respond restrictively to the embeddedwatermark data to set limits on use of the music. Fields in thewatermark can specify any or all of (or others in addition to) (a) thetypes of devices on which the music can be played (b) the types ofdevices on which the music can be recorded; (c) the number of times themusic can be played; (d) the number of times the music can be recorded,etc.

The device restrictions (a) and (b) can be of various types. In someembodiments, the restrictions can identify particular units (e.g. byserial number, registered owner, etc.) that are authorized toplay/record the encoded music. Or the restrictions can identifyparticular classes of units (e.g., battery-powered portable players withmusic memories of less than 50 megabytes, disk-based dedicated musicappliances, general purpose personal computers, etc.) Or therestrictions can identify particular performance quality criteria (e.g.,two channel, 16-bit audio at 44.1 KHz sample rate, or lower quality).

The use restrictions (c) and (d) can likewise be of various types.Examples include “do not copy,” “copy once only,” “unrestricted copyingpermitted,” “play once,” “play N times” (where N is a parameterspecified elsewhere in the watermarked data, or by reference to adatabase indexed by a watermark data field), “unrestricted playingpermitted,” etc.

It is straightforward to design a music appliance to respond to usagelimits of zero (e.g. “do not copy”) and infinity (e.g. “unrestrictedcopying permitted,” and “unrestricted playing permitted”). The devicesimply examines one or more bits in the watermark data, and permits (orrefuses) an operation based on the value thereof

Implementation of the other usage-control restrictions can proceed invarious ways. Generally speaking, the stored music can be altered togive effect to the usage-control restrictions. For example, if the musicis “record-once,” then at the time of recording, the appliance can alterthe music in a fashion indicating that it now has “do not record”status. This alteration can be done, e.g., by changing the watermarkdata embedded in the stored music (or adding watermark data), bychanging other data stored in association with the music, etc. If theoriginal signal is stored (as opposed, e.g., to a streaming signal, suchas an internet or wireless transmission), it too should be so-altered.

Likewise with playback limitations. The number of playbacks remainingcan, e.g., be encoded in an updated watermark in the music, be trackedin a separate counter, etc.

More particularly considering the “copy once” usage restriction, anillustrative embodiment provides two distinct watermark payload bits: a“copy once” bit and a “copy never” bit. When originally distributed(whether by internet, wireless, or otherwise), the “copy once” bit isset, and the “copy never” bit is un-set.

When music encoded in this fashion is provided to a compliant recordingdevice, the device is authorized to make one copy. (A compliant deviceis one that recognizes encoded watermark data, and behaves as dictatedby the watermark.) When this privilege is exercised, the recordingdevice must alter the data to ensure that no further copying ispossible. In the illustrated embodiment, this alteration is effected bythe recording device adding a second watermark to both the music, withthe “copy never” bit asserted. The second watermark must generally beencoded in an “orthogonal” domain, so that it will be detectablenotwithstanding the continued presence of the original watermark.Compliant equipment must then check for both watermarks, and refuse tocopy if either is found to have the “copy never” bit asserted.

One advantage to this arrangement is that if the watermark signal hasundergone some form of corruption (e.g. scaling or resampling), thefirst watermark may have been weakened. In contrast, the secondwatermark will be native to the corrupted signal, and thus be moreeasily detected. (The corruption may also contribute to theorthogonality of one watermark relative to the other, since the twowatermarks may not have precisely the same time base or otherfoundation.)

An alternative approach is not to encode the “copy never” bit in theoriginal music, but leave this bit (in whatever manifestation) blank(i.e. neither “1” nor “0”). In transform-based watermark techniques,this can mean leaving transform coefficient(s) corresponding to the“copy never” bit un-changed. If the watermarking is effected in thetemporal sample domain (or spatial domain, for image data), this canmean leaving certain samples (pixels) unmodified. The recording devicecan then alter the transform coefficients and/or samples as necessary toassert the previously-unencoded “copy never” bit when the permittedrecording is made.

In such a system, compliant recording devices check for the “copy never”bit in the sole watermark, and refuse to make a copy if it is asserted(ignoring the value of any “copy once” bit).

A third approach to “copy once” is to set both the “copy once” and “copynever” bits, but set the former bit very weakly (e.g. using lower gainand/or high frequency DCT coefficients that do not survive certainprocessing). The frail “copy once” bit is designed not to survive commoncorruptions, e.g., resampling, scaling, digital to analog conversion,etc. To further assure that the “copy once” bit is lost, the recordingdevice can deliberately add a weak noise signal that masks this bit(e.g. by adding a noise signal in the frequency band whose DCTcoefficient conveys the “copy once” bit). In contrast, the “never copy”bit is unchanged and reliably detectable.

In such a system, compliant devices check for the “copy once” bit in thesole watermark, and refuse to make a copy if it is not detected as set.

These three examples are but illustrations of many possible techniquesfor changing the rights associated with a work. Many other techniquesare known. See, e.g., the proposals for watermark-based copy controlsystems for digital video at the Copy Protection Technical WorkingGroup, http://www.dvcc.com/dhsg/, from which certain of the foregoingexamples are drawn. See also Bloom et al, “Copy Protection for DVDVideo,” IEEE Proceedings, Special Issue on Identification and Protectionof Multimedia Information, June, 1999.

Scalability

One feature that is desirable in many detector embodiments isscalability. This refers to the ability of a detector to scale itscomputational demands to match the computational resources available toit. If a detector is running on a high performance Pentium IIIworkstation, it should be “doing more” than if the same detector isrunning on a slow microcontroller. One way scalability can be achievedis by processing more or less chunks of input data (e.g. temporalexcerpts of music, or blocks/macroblocks of pixels in a frame of videodata) to decode watermarks. For example, an input audio stream might bebroken into chunks of one second each. A fast processor may completedecoding of each chunk in less than a second, permitting it successivelyto process each chunk in the data stream. In contrast, a slow processormay require two and a half seconds to decode the watermark from a chunk.While it is processing a first chunk, the second and third pass byun-decoded. The processor next grabs and processes the fourth chunk,permitting the fifth and sixth to pass by un-encoded.

The detector running on the fast processor is clearly more difficult to“fool,” and yields a decoded watermark of higher confidence. But bothsystems decode the watermark, and both operate in “real time.”

The skipping of input data in the temporal (e.g. music or video) orspatial (e.g. image or video) domain is but one example of howscalability can be achieved. Many other approaches are known to thoseskilled in the art. Some of these alternatives rely on spending more orless time in the data analysis phases of watermark decoding, such ascross-correlation operations.

Reference has been made to watermarked UIDs as referring to a databasefrom which larger data strings (e.g. web addresses, musician names,etc.) can be retrieved. In some embodiments, the data record referencedby a UID can, in turn, point to several other database records. By sucharrangements, it is often possible to reduce the payload of thewatermark, since a single UID reference can lead to several differentdata records.

Production Tools

In the prior art, the watermark embedded in a source material istypically consistent and static through a work—unchanging from beginningto end. But as will be recognized from the foregoing, there are manyapplications that are better served by changing the watermark datadynamically during the course of the work. According to another aspectof the invention, a production tool is provided that facilitates theselection and embedding of dynamically-changing watermark data. One suchembodiment is a software program having a user interface thatgraphically displays the different watermark fields that are beingembedded in a work, and presents a library of data (textually or byicons) that can be inserted into each field, and/or permits the user totype in data to be encoded. Another control on the UI controls theadvance and rewind of the media, permitting the user to determine thelocation at which different watermark data begins and ends. Graphicalparadigms known from video- and audio-editing tools can be used toindicate the starting and ending frames/samples for each differentwatermark payload.

Such a tool can be of the standalone variety, or can be integrated intothe desktop audio- and video-production and editing tools offered byvendors such as Avid, Adobe, Jaleo, Pinnacle Systems, SoundForge, SonicFoundry, Xing Technology, Prosoniq, and Sonic Desktop Software.

Payment-Based Systems

Another aspect of the present invention is the use of anonymous paymenttokens that can be used to obtain content on the web. In one embodiment,a token comprises a 128-bit pseudo-random number, to which additionalbits identifying an issuing bank (or other issuing institution) areappended. (The additional bits can be the IP address of a web server ofthe bank, a routing number identifying the bank for electronic wiretransfers, or other identifier.) The 128-bit numbers are randomlygenerated by the bank—commonly as needed—and each represents a fixedincrement of money, e.g. ten cents.

A consumer wishing to have a store of currency for such commerce paysthe bank, e.g., $10 in exchange for 100 tokens. These tokens aretransferred electronically to disk or other storage in the consumer'scomputer in response, e.g., to a credit card authorization, or may beprovided by diskette or other storage medium over the counter at a bankbranch (in which case the consumer thereafter copies the numbers intostorage of his or her computer). (Outlets other than banks can of coursebe employed for distributing such numbers, much in the manner thatconvenience and many grocery stores commonly issue money orders.)

Imagine that the consumer wishes to view the final quarter of aTrailblazer basketball game that aired on television a week ago. (Theconsumer may have either missed the game, or may have seen it but wantsto see the last quarter again.) The user directs a web browser to a website maintained for such purpose and performs a search to identify thedesired program. (Typically, the web site is maintained by theproprietor that holds the copyright in the material, but this need notbe the case. Some material may be available at several web sites, e.g.,maintained by ABC Sports, the National Basketball Association, andSports Illustrated.) The search can use any of various known searchengines, e.g., Infoseek, Verity, etc., and can permit searching by titleterms, keywords, date of airing, copyright owner, etc. By typing in,e.g., the keyword ‘Trailblazers’ and the date ‘Apr. 26, 1999,’ theconsumer is presented a listing of videos available for download. One,hopefully, is the requested game. With each listing is an indication ofan associated nominal charge (e.g. 80 cents).

On clicking on a hypertext link associated with the desired basketballgame, the viewer is presented a further screen with one or more options.The first of the listed options is the entire game, with commercials.The charge is the nominal charge presented on the earlier screen (i.e.80 cents). Other options may include the first, second, third, andfourth quarters of the game individually, each of which—save the last,costs 20 cents. The last may be charged at a premium rate, e.g., 30cents. Clicking on the desired video option yields a further screenthrough which payment is effected.

To pay for the requested video, the consumer instructs his or hercomputer to transfer three of the earlier-purchased tokens over the webto the video provider. Various user interface metaphors can be employedto facilitate this transfer, e.g., permitting the user to type theamount of money to be transferred in a dialog box presented on-screen,or dropping/dragging icons representing tokens from an on-screen“wallet” to an on-screen “ticket booth” (or over an icon or thumbnailrepresenting the desired content), clicking on an “increment” counterdisplayed adjacent the listing of the content, etc. Once the consumerhas authorized a transfer of sufficient tokens, the consumer's computersends to the web site (or to such other web address as HTML encoding inthe viewed web page may indicate) the tokens. This transmission simplytakes the form of the three 128+ bit numbers (the ‘+’ indicating thebank identifier)—in whatever packet or other format may be used by theinternet link. Once dispatched in this manner, the tokens are deletedfrom the user's computer, or simply marked as spent. (Of course, inother embodiments, a record of the expenditure may be stored in theconsumer's computer, e.g., with the token contents and a record of theaudio or video purchase to which they were applied.)

Since the amount of money is nominal, no encryption is provided in thisembodiment, although encryption can naturally be provided in otherembodiments (e.g., either in sending the tokens from the user to the website, or earlier, in sending the tokens to the user). As will be seen,provided that the media provider immediately sends the tokens to thebank in real time, encryption is a nice feature but not mandatory

On receipt of the token data, the web site immediately routes the tokendata to the identified bank, together with an identifier of the mediaprovider or account to which the finds represented thereby are to becredited. The bank checks whether the 128-bit numbers have been issuedby that bank, and whether they have already been spent. If the numbersare valid, the bank updates its disk-based records to indicate that thethree tokens have been spent and that the bank now owes the mediasupplier 30 cents, which it may either pay immediately (e.g., bycrediting to an account identified by the media provider) or as one lumpsum at the end of the month. The bank then sends a message to the website confirming that the tokens were valid and credited to the requestedaccount. (Optionally, a message can be sent to the purchaser of thetokens (if known), reporting that the tokens have been redeemed.)

In response, the web site begins delivery of the requested video to theconsumer. In the illustrated embodiment, the video is watermarked priorto delivery, but otherwise sent in unencrypted fashion, typically instreaming format, but optionally in file format. (Encryption can be usedin other embodiments.) The watermarking in the illustrated embodiment isaccomplished on-the-fly and can include various data, including the dateof downloading, the download site, the destination IP address, theidentity of the purchaser (if known), etc.

The large size of the video and the small charge assessed thereforprovide disincentives for the consumer making illicit copies.(Especially as to archival material whose value decays with time, thereis not much after-market demand that could be served by illicit copies,making third party compilation of such material for re-distributionfinancially unattractive. First run video, and material that keeps ahigh value over time, would not be as well suited for such distribution,and could better employ technology disclosed elsewhere herein.)

In some embodiments, the integrity of the received video is checked onreceipt. This feature is described below in the section entitledWatermark-Based Receipts.

In the illustrative system, nothing in the tokens indicates the identityof the purchaser. The web site knows the IP address of the site to whichvideo was delivered, but need not otherwise know the identity of thepurchaser. The bank would probably maintain a record of who purchasedthe tokens, but need not. In any event, such tokens could thereafter beexchanged among consumers, resulting in anonymity from the bank, ifdesired.

As described above, the video excerpts from which the consumer canselect include commercials. At some sites, video may be provided fromwhich the commercials have been excised, or which is delivered in amanner that skips past the commercials without transmitting same to theconsumer. Such video will naturally command a premium price. In someembodiments, the difference in price is electronically credited ascompensation to accounts maintained for (or by) the advertisers, whoseadvertisements are not being viewed by such consumers. (Theidentification of advertisers to be credited is desirably permanentlyencoded in the video, either throughout the video (if the video has hadthe commercials removed therefrom), or by data in the commercialsthemselves (which commercials are skipped for transmission to theconsumer, but can still be decoded at the video head-end. Such encodingcan be by in-band watermarking or otherwise.)

While the foregoing discussion particularly considered video as thedesired content, the same principles are equally applicable inconnection with audio, still imagery, and other content.

The token-based payment method is but one of many that can be employed;the literature relating to on-line payment mechanisms is extensive, andall such systems can generally be here-employed.

Tracking 128-bit tokens can be a logistical problem for the bank. Oneapproach is to have a memory with 10¹²⁸ locations, and at each locationstore a two-bit value (e.g. 00=never issued; 01=issued but not spent;10=issued and spent; 11=reserved). More complete data couldalternatively be stored, but such a memory would be impractically large.

One alternative approach is to hash each 128-bit number, when issued, toa much smaller key value (e.g. 20 bits). A memory with 10²⁰ locationscan be indexed by this key. Each such location can include four data: anissued 128-bit token number that hashes to that value, first and seconddate fields indicating the date/time on which that token was issued andredeemed, respectively, and a link specifying the address of a nextmemory location. That next memory location (outside of the original 10²⁰locations) can include four more data, this time for a secondissued-128-bit token number that hashed to the original key value, twodate fields, and again with a link to a subsequent storage location,etc.

When 128-bit random number is generated, the original memory locationindexed by the hash code of that number is checked for an earlier numberof the identical value (to avoid issuance of duplicate tokens). Eachsuccessive location in the linked chain of memory locations is checkedfor the same 128-bit number. When the end of the linked chain isreached, the bank knows that the 128-bit random number has notpreviously been issued, and writes that number in the last-addressedlocation, together with the date of issuance, and a link to a nextstorage location.

When a 128-bit token is received, the same linked-list processing occursto identify a first location and to thereafter step through eachsubsequent location until a match is found between the token number andthe number stored in one of the linked memory locations. When found,that number is marked as redeemed by writing a redemption date/time inthe corresponding field.

If the search reaches the end of the linked chain without finding amatch between the stored numbers and the token number, the token istreated as invalid (i.e. not issued by that bank).

Other manners of tracking the large number of possible token numbers canof course be used: the foregoing is just exemplary. Or the tokensneedn't be tracked at all. Such an arrangement is highly practical ifthe token has sufficient bits. With the illustrated 128 bits, forexample the chance of two identical tokens being issued isinfinitesimally small, so checking for duplicate issuance can be omittedif desired. In such case, the bank can simply maintain an ordered listof the token numbers still outstanding and valid. As new tokens aredispensed, their token numbers are added to the list. As tokens areredeemed, their numbers are deleted from the list known list processingtechniques can be employed to speed such search, update, and deleteactions.

Watermark-Based Receipts

Pay-for content applications commonly assume that if content istransmitted from a server (or head-end. etc.). it is necessarilyreceived. Sometimes this assumption is wrong. Network outages andinterruptions and internet traffic load can diminish (e.g., droppedvideo frames) or even negate (e.g. failed delivery), expected consumerenjoyment of content. In such cases the consumer is left to haggle withthe content provider in order to obtain an adjustment, or refund ofassessed charges.

Watermarks provide a mechanism for confirming receipt of content. If awatermark is detected continuously during a download or other deliveryevent, a software program (or hardware device) can issue an electronicreceipt attesting that the content was properly delivered. This receiptcan be stored, and/or sent to the content distributor to confirmdelivery.

In one embodiment, a content receiving device (e.g., computer,television or set-top box, audio appliance, etc.) periodically decodes awatermark from the received content to confirm its continued reception.For example, every five seconds a watermark detector can decode thewatermark and make a record of the decoded data (or simply record thefact of continued detection of the same watermark). When a changedwatermark is detected (i.e., reception of a different content objectbegins), the duration of the previously-received content is logged, anda receipt is issued.

In a related embodiment, the last portion (e.g., 5 seconds, frame, etc.)of the content bears a different “end of content” watermark thattriggers issuance of a receipt. Such a watermark can indicate the lengthof the content, to serve as a cross-check against the periodic watermarkpolling. (E.g., if periodic sampling at 2 second intervals yields 545samples corresponding to the same content, and if the “end of content”watermark indicates that the content was 1090 seconds long, then receiptof the entire content can be confirmed.)

In another embodiment, the watermark can change during the course of thecontent by including, e.g., a datum that increments every frame or otherincrement of time (e.g., frame number, time stamp, etc.). A watermarkdetector can monitor the continued incrementing of this datum throughoutthe content to confirm that no part was garbled (which would destroy thewatermark) or was otherwise missing. Again, at the end of delivery, thereceiving system can issue a confirmation that XXX frames/seconds/etc.of the identified content were received.

One application of such technology is to bill for content based onreceipt, rather than transmission. Moreover, billings can be adjustedbased on percentage of content-value received. If delivery isinterrupted mid-way through (e.g., by the consumer disabling thecontent-receiving device), the nominal billing for the content can behalved. Some prolonged content, e.g., televised/web-broadcast universityclasses, cannot be “consumed” in one session, and are thus particularlywell suited for such pay-as-you-consume billing.

Another application of such technology is in advertising verification.Presently, ads are tracked by transmission or, less frequently, bydetection of an embedded code on receipt (c.f., Nielsen Media Research'sU.S. Pat. Nos. 5,850,249 and 5,737,025). However, suchreception-detectors—once triggered—generally do not further note thelength of time that the advertising was received, so the same data isproduced regardless of whether only five or fifty seconds of acommercial is presented. Watermark monitoring as contemplated hereinallows the duration of the advertising impression to be preciselytracked.

In one application of this technology, recipients of advertising areprovided incentives for viewing advertising in its entirety. Forexample, a content-receiving device can include a watermark detectorthat issues a receipt for each advertisement that is heard/viewed in itsentirety. These receipts can be redeemed, e.g., for content tokens asdescribed elsewhere herein, for monetary value, etc. In someembodiments, receipts are generic and can all be applied to a desiredpremium, regardless of the advertisements through which they wereearned. In other embodiments, the receipts are associated with theparticular advertisers (or class of advertisers). Thus, a TV viewer whoaccumulates 50 receipts from advertising originating from Procter &Gamble may be able to redeem same for a coupon good for $2.50 off anyProcter & Gamble product, or receipts from Delta Airlines may beredeemed for Delta frequency flier miles (e.g., at a rate of one mileper minute of advertising). Such incentives are particularly useful innew forms of media that give the consumer enhanced opportunities tofast-forward or otherwise skip advertising.

(Although the foregoing “receipt” concept has been described inconjunction with watermark data (and use of watermark technology isbelieved to be inherently advantageous in this application), the sameprinciples can likewise be implemented with ancillary data conveyed byother means.)

Master Global Address

As suggested above, it is desirable that each piece of content have aweb address (the “Master Global Address” (MGA), or “Master IP Address”)associated with it. Such address is typically conveyed with the content,e.g., by an IP address watermarked therein.

Consider a consumer who downloads a streaming video having an Englishlanguage soundtrack. The viewer may not speak English, or may otherwiseprefer to listen to the soundtrack in another language. The user candecode the watermark data embedded in the video and initiate a link tothe associated web address. There the user is presented with a list ofsoundtracks for that content object in other languages. The viewer canclick on the desired language and receive same via a second simultaneoustransmission (e.g., a second socket channel). The consumer's audio/videoappliance can substitute the desired audio track for the default Englishtrack.

If the streaming video and the alternative soundtrack are hosted on thesame server, synchronization is straightforward. The process governingtransmission of the alternative soundtrack identifies the process thatis streaming video to the same IP address. Based on SMPTE, or othertime/frame data, the former process syncs to the latter. (If the twodata streams don't originate through the same server, time/frame datacan be relayed as necessary to the alternative soundtrack server toeffect synchronization.)

Another application of the Master Global Address is to serve as a pointto which monitoring stations can report the presence, or passage, ofcontent. Consider, for example, a copyright-aware node through whichcontent signals pass, e.g., a computer node on a network, a satellitetransponder, etc. Whenever the node detects passage of a media object(e.g., by reference to a file extension, such as MP3, JPG, AVI, etc.),it sends a “ping” over the internet to the address encoded in theobject, simply reporting passage of the object. Similar monitoringfacilities can be provided in end user computers, e.g., reportingFileOpen, FileSave, Printing, or other use of content bearing MGA data.

This system can be expanded to include “ping” and “pong” phases ofoperation. When a software application (or a user appliance, such as avideo or audio playback device) encounters a media object (e.g., at timeof file open, at time of playback, etc.), it pings the MGA site toreport the encounter. The MGA site “pongs” back, responding withinstructions appropriate to the encounter. For example, if the objectrequires payment of a fee before full functionality or access is to begranted, the MGA site can respond to the application with instructionsthat the object be used (e.g., played back) only in some crippled statepreventing the user's full enjoyment (e.g., impaired resolution, orimpaired sound quality, or excerpts only, etc.). The MGA site can alsoinform the user application of the terms (e.g., payment) by which fullfunctionality can be obtained. The application can graphically oraudibly present such information to the user, who can authorize apayment, if desired, so that the content can be enjoyed in a less- (orun-) crippled state. On receipt of the payment authorization, the MGAsite can inform the user application that enhanced access/usage rightshave been purchased, and that the application may proceed accordingly.

Yet another application of the MGA is to present the user of a contentobject a menu of options that is customized to that object.

In current graphical operating systems, when a user clicks on an icon(e.g., with the right mouse button), a menu is presented detailingactions that can be undertaken in connection with the icon, or the filerepresented thereby. Such options are pre-programmed (i.e., static), andare typically determined by the operating system based solely on thefile extension.

In accordance with this aspect of the present invention, clicking on anicon representing a media object initiates an internet link to the MGAsite associated with the object. The MGA site responds with data that isused to customize the menu of options presented to the user inconnection with that particular object.

Consider an icon representing a JPG image file. Right-clicking on theicon may yield a menu that gives the user various options presented bythe operating system (e.g., delete, compress, rename), and additionaloptions customized in accordance with data from the object's MGA site.These customized options may include, e.g.,

-   -   (a) open in 100×150 pixel format for free;    -   (b) open in 480×640 pixel format for ten cents;    -   (c) open in 960×1280 pixel format for twenty cents;    -   (d) purchase rights to use this image in a newsletter having a        circulation of under 1000 for $1.25;    -   (e) display a complete listing of license options.

Clicking on options (b) or (c) initiates a commerce application throughwhich funds are electronically transferred to the MGA site (by theabove-described tokens or otherwise). In response, the MGA site responds(e.g., with TCP/IP or HTML instructions) authorizing an application onthe user's computer to open the file in the requested manner. (Thedefault application for JPG applications can then automatically belaunched, or the computer may first query the user whether anotherapplication should be used instead.)

Clicking on option (d) proceeds as above, and permits full use of theimage on the computer. Moreover, the MGA site sends a digitalcertificate to the user's computer memorializing the usage rightspurchased by the consumer.

In this particular arrangement, no access control is placed on thecontent, e.g., by encryption, secure container technology, or the like.The nominal fees, and the ease of licensing, make it simple for the userto “do the right thing” and avoid copyright liability. In otherembodiments, of course, known access control techniques can be used tolimit use of the object until the requisite payment has been made.

Naturally, records of all such transactions are also logged at the MGAsite.

Clicking on option (e) opens a browser window on the user's computer toa web site that presents a complete listing of license options availablefor that image. (The address of this web site is included incustomization data relayed to the user device from the MGA site, but notexplicitly shown to the user on the menu.) Through such web site, theuser can select desired rights, effect payment, and receive thenecessary authorization for software applications on the user's computer(or other media appliance) to open and/or process the content.

The object on which the user “clicks” needn't be an icon. It can be animage or other graphical representation. (And a “click” isn't necessary;a voice command or other signal may be used to the same effect with anaudio clip or selection.)

Consider the popular merchandising of books and CDs over the internet. AJPG or other image file depicting the cover of a book, or the artwork ofa CD cover, can be treated as a media object, and can include awatermarked MGA pointer. Right-clicking on such an image of a book covercould, through the MGA site, present to the user a menu of options thatincludes—in addition to those normally presented in conjunction with aJPG file—the following:

-   -   (a) “See the review of this book published in the New York Times        on Apr. 19, 1999”    -   (b) “See the list of reviews of this book at Amazon.com”    -   (c) “Enter your own review of this book, for posting on        Amazon.com”    -   (d) “See today's sales rank of this book at Amazon.com”    -   (e) “Purchase this book from Amazon.com for $16.95”    -   (f) “Purchase this book from Barnesandnoble.com for $19.95 and        receive a $5.00 credit towards your next purchase”    -   (g) “Link to the web site that tells about the release of this        title as a motion picture (presently scheduled to open on Oct.        10, 1999)”    -   (h) “Link to the Yahoo listing of web sites relating to this        book”    -   (i) “Search Lycos for listings relating to this book.”

If the user selects one of the purchase options from the menu, apre-stored e-commerce profile—containing the user name, credit cardnumber, billing address, ship-to address, etc., possibly in the form ofan encrypted object—could be sent to the MGA site (or to the bookseller)to effect the purchase, or such selection could initiate display ofadditional screens or sub-menus through which the user would manuallyenter or select such information for transmission.

Others of the selections cause a new browser window to open on theuser's computer, opening to a URL specified in data relayed from the MGAsite but not displayed to the user in the menu. Appropriate HTMLinstructions can be generated to effect a particular query or otheroperation at the specified URL.

In some embodiments, the customized menu presents only a single choicein addition to those normally provided by the operating system, e.g.,“Link to home.” Clicking on this option opens a browser window to a homepage at the MGA for that object. On that page, the user is presentedwith all of the foregoing options, and more (possibly includingadvertising graphics or multi-media). Such objects can serve as powerfulmarketing agents. Returning to the example discussed above, a JPG imagefile of a book cover may have, as its MGA, a web page hosted by aparticular bookseller, providing purchase options and other informationfor that book. Marketing of books (or CDs, or cars, or consumerappliances, or virtually anything else) can be effected by disseminatingsuch vendor-issued JPGs as widely as possible. Some book cover JPGs maybe distributed by Amazon.com, others by Barnes&Noble.com, others byBorders.com—each pointing back to a different MGA through which purchasetransactions for that book may be performed.

Returning to the MGA-customized menus, these needn't be limited to menusresulting from clicking on an icon or image (or signaling during anaudio excerpt). Drop-down menus in application programs can likewise bepopulated with customized options, in accordance with customization dataobtained from the MGA site for the object presently being accessed orused. Most graphical operating systems and application programs havewell developed toolsets permitting such menu customization. Again, otherdata relayed from the MGA site is not shown to the user, but is employedby the computer (e.g., a browser program) to carry out menu optionsselected by the user.

Again the foregoing techniques are equally applicable for still images,audio, video, and other forms of content, and can readily be adapted foruse both with general purpose computers, software applications, andspecialized media appliances.

While, for expository convenience, the foregoing discussion contemplatedembedding a literal URL address in the object as the MGA, more typicallythis is not the case. Instead, the MGA more commonly comprisesidentification data for the object (e.g. a 128-bit random ID), togetherwith the URL for a name server computer that serves many (perhapsmillions) of such objects (an example of the latter is the DigimarcMarcCentre server).

To obtain the desired data as detailed above, the user's computer(sometimes termed a client computer) links to the name server computerand provides the ID of the object being processed. The name servercomputer uses this ID to query a database, and obtains from the databasethe current IP address to which such queries should be routed. The nameserver computer can relay the request from the client computer to thecorrect destination address, or can return the correct destinationaddress to the client computer, which can initiate such a link itself.By such arrangement, the IP address ultimately associated with an objectcan be easily changed as needed, simply by changing the correspondingrecord in the name server database, without rendering obsolete legacyobjects having out-of-date addresses encoded therein.

In some embodiments, the URL of the name server needn't be included inthe watermark. In the absence of a specified URL, the client computermay direct such links to a default name server address instead (storedlocally or remotely). If that server doesn't recognize the object ID, itcan return an error code, or pass the query on to other name servers.Those servers, in turn, can pass the query along to still other nameservers if they don't recognize the object ID. In this fashion, anexponentially-large number of name servers might be quickly polled forinformation relating to the identified object. Alternatively, ratherthan encoding the complete IP address of the name server in an objectwatermark, the first N (e.g., 16) bits of the object ID might be used asa short-hand for one of 65,536 predetermined name server addresses, inaccordance with data stored locally (e.g., on RAM or disk in the user'scomputer) or remotely (e.g., at a default name server IP address).

While the basic concept idea behind embedding MGA data within an objectis to point to a repository of data about the object, a pointer theother way may be achieved as well.

As noted, the “ping” application of MGA data permits an MGA site to beinformed of sites through which its object passes. More generally, theMGA site can log the originating address of each query it receives. Eachsuch address can be presumed to have (or have had) a copy of thecorresponding object. Media owners can thereby track the disseminationof copies of their media objects—at least insofar as use of such objectsentails communicating with the associated MGA site.

Such tracking offers a great number of opportunities, some in the areaof commerce. The MGA site corresponding to the cover art of a GarthBrooks CD, for example, can provide a listing of IP addresses of personsinterested in that CD. Email or promotional data objects (e.g., audioclips) can be sent to that list of addresses when a subsequent GarthBrooks CD is released.

Such tracking also opens up a new dimension of internet searching.Presently, internet search engines use a brute force approach, visitingmillions of pages across the web in order to identify, for example, adozen instances of a given photograph file. MGAs offer a shortcut tosuch brute force approaches. With the present technology, a searchengine can find a single instance of a photograph file and, by detectionof the MGA data watermarked therein, link to the corresponding MGA site.From the MGA site, the search engine can obtain a listing (if suchqueries are authorized) of some or all of the other sites known by theMGA site to have copies of that photograph file. (Providing such data tosearch engines is a commerce opportunity for such MGA sites, which maypermit such access to its listing of sites only in exchange for a fee.Or the MGA site may arrange to collect a tribute payment from the searchengine proprietor each time the engine responds to a user query usingdata collected from the MGA site.)

Many of the addresses logged by the MGA may not be publicly-accessibledata stores. The search engine can check each listed address to ensurethat the desired object is present and accessible before adding theaddress to its database.

Covert Tracing

Application Ser. No. 09/185,380 (now U.S. Pat. No. 6,549,638) describesanti-counterfeiting technology that looks for the presence of digitaldata corresponding to bank note imagery in a computer system, and makesa covert record of any attempt to process such data (e.g., Scan,FileOpen, FileSave, Print, Edit, etc.). Such records are hidden from theuser of the system (using, e.g., various data encryption and obscuringtechniques), but authorized law enforcement officials are provided toolsby which these records can be recovered. The forensic data therebyobtained may prove useful in prosecuting counterfeiters. (Knowledge thata computer may be covertly storing evidence of attempted counterfeitingactions may prove as, or more, valuable in deterring counterfeiting thanthe covert records themselves.)

The same techniques can be employed to deter unauthorized processing ofaudio, image, video, or content by media pirates. In one embodiment, acomputer's operating system (including peripheral device drivers)monitors various data within the system (e.g., data sent to writeablestorage media, or sent via a serial port or network connection, etc.)for data bearing a do-not-copy watermark. The presence of such databeing sent, e.g., to a writeable disk or to a remote computer, indicatesthat the do-not-copy instruction has been circumvented. In such case,the operating system writes one or more covert records memorializing theactivity, for possible use in criminal prosecution if the computer islawfully seized.

The example just-provided is but one of many monitoring and responsetechniques that may be employed to deter circumvention ofcopy-protection or other access control systems. Generally speaking, ifcontent data is found where it shouldn't be, or is found used as itshouldn't be used, a corresponding record should be made. (Otherintervention actions can be triggered as well; covert tracing isdesirably just one of several parallel responses to suspected hacking.)

Meta-Data Accessed Using Watermarks

Meta-data, in formats known as XML, SGML, and HTML, is widely used tocommunicate information about digital objects (e.g., author, keywords,price, rights, caption, etc.). More generally, meta-data can be thoughtof as any data construct which associates the name of a property (e.g.,“author), with the value of the property (e.g., “Mark Twain”). Such datacommonly appears in a tag format, such as the following:<META NAME=“author” CONTENT=“Mark Twain”>

Meta-data is commonly exchanged between server and client computers inconjunction with the digital objects to which they relate (e.g., thetext of a Mark Twain book).

As detailed herein, an important application of watermarking is likewiseto convey information about media—in this case embedded within the mediacontent itself (e.g., providing unique identification, establishing somebasic behaviors such as do not copy, and providing links to extendedfunctionality).

For meta-data to be useful, it must be linked to associated content,whether in the context of a browser, application program, operatingsystem, asset management system, search engine, etc. However, asdetailed below, the content and the associated meta-tags needn't alwaysbe conveyed together.

Consider an application program or other client process that receives awatermarked media object. The watermark includes an MGA for that object(which, as noted above, may not specify an ultimate IP address). Storedat the MGA site is meta-data corresponding to the object. By linking tothe MGA site identified by the object's watermark, the client computercan obtain the meta-data corresponding to the object. This data can bestored at the client computer and used just as any other meta-data,e.g., to define the local functions that should be available for usewith that object (e.g., buy, search, etc.)

A particular example is an on-line catalog of stock photography. Eachphotograph is watermarked with MGA data. To identify the photographer,copyright date, price, telephone number, subject, etc., an applicationprogram can link to the MGA site for that photograph, and obtain thecorresponding meta-data. This data can then be displayed or used asneeded. Data objects of disparate formats thus can readily be handledwithin a single, simple application program, since the program needn'tconcern itself with the varying formats for the associated meta-data(assuming the name servers provide this data in standardized format).Substantial flexibility in programming and object formatting is therebyachieved.

Returning to the internet search engine example described above, MGAsmay become recognized as repositories rich in meta-data for mediaobjects. Specialized search engines may focus their data collectionaround such sites, and be able to quickly identify the MGA sitescorresponding to various Boolean combinations of meta-tag parameters.

Asset Management/Containers

Much has been written on the topic of asset rights management. Samplepatent documents include U.S. Pat. Nos. 5,892,900, 5,715,403, 5,638,443,5,634,012, 5,629,980 and laid-open European application EP 862,318. Muchof the technical work is memorialized in journal articles, which can beidentified by searching for relevant company names and trademarks suchas IBM's Cryptolope system, Portland Software's ZipLock system, theRights Exchange service by Softbank Net Solutions, and the DigiBoxsystem from InterTrust Technologies.

An exemplary asset management system makes content available (e.g. froma web server, or on a new computer's hard disk) in encrypted form.Associated with the encrypted content is data identifying the content(e.g. a preview) and data specifying various rights associated with thecontent. If a user wants to make fuller use of the content, the userprovides a charge authorization (e.g. a credit card) to the distributor,who then provides a decryption key, allowing access to the content.(Such systems are often realized using object-based technology. In suchsystems, the content is commonly said to be distributed in a “securecontainer.”)

Desirably, the content should be marked (personalized/serialized) sothat any illicit use of the content (after decryption) can be tracked.This marking can be performed with watermarking, which assures that themark travels with the content wherever—and in whatever form—it may go.The watermarking can be effected by the distributor—prior todissemination of the encrypted object—such as by encoding a UID that isassociated in a database with that particular container. When accessrights are granted to that container, the database record can be updatedto reflect the purchaser, the purchase date, the rights granted, etc. Analternative is to include a watermark encoder in the software tool usedto access (e.g. decrypt) the content. Such an encoder can embedwatermark data in the content as it is released from the securecontainer, before it is provided to the user. The embedded data caninclude a UID. This UID can be assigned by the distributor prior todisseminating the container. Alternatively, the UID can be a data stringnot known or created until access rights have been granted. In additionto the UID, the watermark can include other data not known to thedistributor, e.g. information specific to the time(s) and manner(s) ofaccessing the content.

As noted earlier, access rights systems can be realized with watermarkswithout containers etc. For example, in a trusting world, copyrightedworks can be freely available on the web. If a user wishes to makelawful use of the work, the user can decode its watermark to determinethe work's terms and conditions of use. This may entail linking to a website specified by the embedded watermark (directly, or through anintermediate database), which specifies the desired information. Theuser can then arrange the necessary payment, and use the item knowingthat the necessary rights have been secured.

Remote Reconfiguration of Watermark Detectors

In some cases, it is desirable to reconfigure watermark detectorsremotely. Such functionality is desirable, for example, if a watermarksystem is hacked or otherwise compromised.

In accordance with this aspect of the present invention, some aspect ofa watermark detector's operation is changed in response to a command.The change can take various forms. In watermark systems employingpseudo-random key data (e.g., spread spectrum spreading signals), thepseudo-random signal used for detection can be changed. In systems usingDFT processing, the mapping between message bits and DFT coefficientscan be changed. In still other systems, the decoding can proceed asbefore, but the significance of one or more bits can be changed (e.g.,bits that were normally interpreted as defining Field A can beinterpreted as defining Field B, and vice versa). In yet other systems,the decoding can proceed as before, but the response of a device to agiven watermark signal can be changed. In still other systems, a set ofsoftware instructions can be re-written or re-ordered to effect a changein detector operation.

The command can be conveyed in various ways. In one embodiment, it canbe a trigger bit in the watermark payload. Normally the bit has a valueof “0.” If the bit has a value of “1,” the detector system responds bychanging its operation. A trigger pattern can also be established, sothat detection of a certain combination of bits in the watermark payloadserves to trigger the change. Reserved states of certain data fields areexamples of patterns that might be employed.

The command can also be conveyed through another channel different thanthe watermark channel (e.g., an SCA channel of an FM broadcast, or thesub-titling data channel of video broadcasts, or header data within anMPEG data stream, etc., etc.).

The change can proceed in accordance with a pre-programmed rule (e.g.,codes progressing successively through a numerically oralgorithmically-determined progression), or the change can proceed inaccordance with data specified elsewhere in the payload of the watermarkbearing the trigger bit (e.g., instead of being interpreted in normalfashion, the non-trigger bits of the detected watermark can define a newpseudo-random key data. Or the change can proceed in accordance withdata conveyed in successively-presented watermark payloads, as might bedone in video encoding where each frame of video can convey furtherwatermark information. (This latter arrangement is one offering ahigh-bandwidth re-programming channel through which, e.g., extensivefirmware instructions might be transferred to the detector to replaceinstructions earlier stored.)

By such arrangements, greatly increased detector versatility andfunctionality can be achieved.

Concluding Remarks

Many diverse embodiments are reviewed above—each with a unique set offeatures. (Still others are disclosed in the assignee's patentsincorporated by reference.) To provide a comprehensive disclosurewithout unduly lengthening the specification, applicants incorporate byreference the patents and patent applications referenced above.

This specification should be construed as explicitly teaching thatfeatures illustrated in one such embodiment can generally be used inother embodiments as well. Thus, for example, a date field was notparticularly discussed in connection with payload data for videowatermarking. Nor were “play once” watermarks so-considered. Theinclusion of a calibration signal with (or as part of) the watermark isshown in embodiments of the issued patents, but is not belabored in theabove-described embodiments. Likewise with “simple universal codes.” Thepre-stored commerce profile described in one of the foregoingembodiments is equally applicable to other embodiments as well.Likewise, the presentation of advertising was discussed in connectionwith one embodiment but not others, although it, too, is generallyapplicable. All of these concepts are familiar at Digimarc and areregarded as generally applicable throughout the work expressed inDigimarc's patent disclosures. Practicality prevents an exhaustiverecitation of each individual permutation and combination.

Having described and illustrated the principles of our invention withreference to illustrative embodiments, it will be apparent that thedetailed arrangements can be modified in arrangement and detail withoutdeparting from such principles.

For example, while reference has been made to various uses of wireless,it should be understood that such reference does not just cover FMbroadcast, and wireless internet networking and the like, but alsoincludes other wireless mechanisms. Examples include cell phones anddirect satellite broadcast.

Likewise, while certain embodiments were illustrated with a watermarkpayload of 100+ bits, in other systems much smaller (or sometimeslarger) payloads are desirable—sometimes as small as 1-8 bits.

While the foregoing examples have each been illustrated with referenceto a particular media type (e.g., video, audio, etc.), it will berecognized that the principles of each embodiment find application withthe other media types as well.

Certain of the appliances contemplated above require user interfacesmore sophisticated than are presently typical on such devices. Thesimplicity of the underlying audio appliance can be preserved, in manyinstances, by using a palmtop computer—coupled by infrared orotherwise—as a temporary user interface to the appliance. Some of theprocessing capability can likewise be off-loaded to an ancillarypalmtop. (Palmtop is here meant to refer generally to any pocket-sizeprogrammable computing device.)

Unless otherwise stated, it should be understood that the digital music,video, and imagery contemplated herein is not of any particular form orformat. Audio, for example, can be of various forms, both streaming andnon-streaming, and of various formats (e.g. MP3, MP4, MS Audio, WindowsMedia Technologies, RealAudio, *.WAV, MIDI, Csound, Dolby's AdvancedAudio Codec (AAC), etc.

Having described and illustrated the principles of the invention withreference to illustrative embodiments, it should be recognized that theinvention is not so limited.

For example, while digital watermarking typically does not leave anyhuman-apparent evidence of alteration or data representation, certain ofthe foregoing applications do not require this. The markings used may bevisible, and even conspicuous, without impairing essentialfunctionality. Thus, bar codes, data glyphs, OCR markings, and othermachine-readable indicia may be substituted, depending on the particularapplication requirements.

While the detailed embodiments were generally described with referenceto desktop computers, it is recognized that such devices willincreasingly be supplanted by other digital appliances, includinggeneral purpose personal digital assistants, multifunction cell phones,and specialized devices. Moreover, the power and utility of theabove-detailed embodiments and devices can be further enhanced byemploying various wireless communications technologies, including theevolving Bluetooth standard.

The implementation of the watermark encoding and decoding systems isstraightforward to artisans in the field, and thus not belabored here.Conventionally, such technology is implemented by suitable software,stored in long term memory (e.g., disk, ROM, etc.), and transferred totemporary memory (e.g., RAM) for execution on an associated CPU. Inother implementations, the functionality can be achieved by dedicatedhardware, or by a combination of hardware and software. Reprogrammablelogic, including FPGAs, can advantageously be employed in certainimplementations.

It should be recognized that the particular combinations of elements andfeatures in the above-detailed embodiments are exemplary only; theinterchanging and substitution of these teachings with other teachingsin this and the incorporated-by-reference patents/applications are alsocontemplated.

In view of the wide variety of embodiments to which the principles andfeatures discussed above can be applied, it should be apparent that thedetailed embodiments are illustrative only and should not be taken aslimiting the scope of the invention. Rather, I claim as my invention allsuch modifications as may come within the scope and spirit of thefollowing claims and equivalents thereof.

1. In a method that includes sensing audio in a device, generatingidentifier data from the sensed audio, and determining informationrelated to said audio by reference to said identifier data, animprovement comprising: receiving at said device, from a remote computersystem and responsive to information sent by said device to said remotecomputer system, meta data associated with said audio, said meta dataincluding title and artist information; presenting at least certain ofsaid meta data to a user of said device; following a user's interactionwith a user interface associated with said device, sensing a signalindicating the user's interest in particular electronic content forwhich meta data is presented; transmitting data related to said userindication from said device to a remote system; and in response,delivering corresponding content data to a desired computer distinctfrom said device.
 2. The method of claim 1 wherein said device comprisesa portable device.
 3. The method of claim 2 wherein said sensingincludes sensing ambient audio, using a microphone in said portabledevice.
 4. The method of claim 2, wherein said portable device comprisesa cell phone.
 5. The method of claim 1 wherein said transmitted dataalso serves to designate a recipient of said corresponding content data,different from said user, and the method includes delivering saidcorresponding content data to a computer associated with said recipient.6. The method of claim 5 which further comprises selecting saidrecipient from an electronic address book.
 7. The method of claim 1 inwhich said delivered content data comprises a link to a network addressfrom which electronic content can be obtained.
 8. The method of claim 1that includes decoding steganographically embedded information from thesensed audio.
 9. A method comprising: sensing audio of interest at aportable device; sending data related to the sensed audio to a remotecomputer system; receiving at a portable device, from said remotecomputer, meta data associated with the sensed audio, said meta dataincluding title information; presenting at least certain of said metadata on a display of said portable device; following a user'sinteraction with a user interface associated with said device, sensing asignal indicating the user's interest in particular electronic contentfor which meta data is displayed; transmitting data related to said userindication from said portable device to a remote system; and inresponse, delivering corresponding content data to a desired computerdistinct from said portable device.
 10. A method comprising: sensingaudio of interest that is being streamed to a first user device from afirst remote computer; in response to action by a user, sendingidentification data related to the sensed audio from said first userdevice to a second remote computer distinct from the first, to therebycause delivery of corresponding content data to a recipient computer;wherein said delivery occurs while said audio of interest is still beingstreamed to the first user device.
 11. The method of claim 10 in whichsaid recipient computer comprises said first user device.
 12. The methodof claim 10 in which said recipient computer is distinct from said firstuser device.
 13. The method of claim 10 in which the first remotecomputer is distinct from the second remote computer.
 14. The method ofclaim 10 that further includes sending address information for a desiredrecipient to said second remote computer, said recipient being distinctfrom said user.
 15. The method of claim 10 in which said correspondingcontent data comprises a digital music file.
 16. A method comprising:sensing audio of interest at a portable device; sending data related tothe sensed audio to a remote computer system; and receiving at saidportable device, from said remote computer, concert information for anartist associated with said audio.